Mind the Gap in Distilling StyleGANs
نویسندگان
چکیده
StyleGAN family is one of the most popular Generative Adversarial Networks (GANs) for unconditional generation. Despite its impressive performance, high demand on storage and computation impedes their deployment resource-constrained devices. This paper provides a comprehensive study distilling from StyleGAN-like architecture. Our key insight that main challenge distillation lies in output discrepancy issue, where teacher student model yield different outputs given same input latent code. Standard knowledge losses typically fail under this heterogeneous scenario. We conduct thorough analysis about reasons effects identify mapping network plays vital role determining semantic information generated images. Based finding, we propose novel initialization strategy model, which can ensure consistency to maximum extent. To further enhance between present latent-direction-based loss preserves relations space. Extensive experiments demonstrate effectiveness our approach StyleGAN2 StyleGAN3, outperforming existing GAN methods by large margin. Code available at: https://github.com/xuguodong03/StyleKD .
منابع مشابه
Mind the (Intelligibility) Gap
Intelligibility, evolution and emulation are some of the key notions for digital information preservation. In this paper we define formally these notions on the basis of modules and inter-module dependencies. Subsequently, we discuss how we can handle the evolution of modules and dependencies. This work can be exploited for building advanced preservation information systems and registries.
متن کاملMind the Gap!
In 1835, the French naturalist Felix Dujardin started crushing ciliates under the microscope and observed that the tiny cells exuded a jellylike, water-insoluble substance, which he described as a ‘‘gelée vivante’’ and which was eventually christened ‘‘protoplasm’’ by the physician Johann E. Purkinje and the botanist Hugo von Mohl. Fifty years after Dujardin’s observations, the possibility that...
متن کاملMind the Gap
Intraband-gap states in conjugated polymers have captivated the chemistry community for the better part of five decades due to their position within the forbidden band gap, between the valence and conduction bands. In a large part, this fascination was (and still is) aided by the quest to find alternative materials that are easily processable compared to their solid-state semiconductor counterp...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Lecture Notes in Computer Science
سال: 2022
ISSN: ['1611-3349', '0302-9743']
DOI: https://doi.org/10.1007/978-3-031-19827-4_25